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1 Dynamic ModttalMxtry £21 FlexiLlg f Peygistent Agents 

2 

3 The present invention relates to the dynamic deployment 

4 of functionality in software agents p in particular, a 

5 dynamic modular architecture for the agent that supports 

6 deployment of functionality that is not anticipated when 

7 the agent is instantiated- 
8 

9 Agents represent a means of representing a user in the 

10 electronic world, bringing together all the various 

11 functions that the user wants to perform in that 

12 electronic world, including:' 

13 • the transient r anonymous presence of an online search; 

14 • persistent occasional presence of online shopping at a 

15 particular store; 

16 • persistent passive presence of directed marking; 

17 * persistent though temporary realtime presence in an 

18 online game; 

19 and many more. 
20 

21 US Patent Application Publication Number US2DQ2Q62334 to 

22 Hewlett Packard Company, CO, USA, entitled "Dynamic 

23 agents for dynamic service provision' 1 teaches of dynamic 
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2 



4 



agents and a dynamic agent infrastructure that supports 
dynamic behaviour modification of agents. Dynamic agents 

3 carry out application specific actions, which can be 

loaded and modified on the fly, One key limitation of 

5 the approach presented is that it refers to the dynamic 

S deployment of agent behaviour - but for the system to 

7 work as described, that dynamically deployed behaviour 

8 must be predefined (so that individual programs can know 

9 what to expect from others). The term 'dynamic' is thus 

10 misleading. The work could better be described as the 

11 dynamic deployment of statically defined behaviour, one 

12 problem presented by this aspect of US2002062334 is that 

13 software engineering in the large, and in particular, 

14 multi-party and multi-site development is made much 

15 harder by the need to conform to a rigid prespecified 
.16 sets of functionality. It would be advantageous for new 

17 functionality to toe definable dynamically, so that 

18 separate teams can worker in greater isolation, thereby 

19 simplifying problem decomposition. 
20 

21 US20Q2062334 describes dynamic functionality in an agent. 

22 It does not decouple functionality and therefore does not 

23 support functionaliry that is not at least defined at 

24 agent start-up. It discloses that the dynamic agent's 

25 modifiable portion includes its data, knowledge and 

26 action programs, which determines its application 

27 specific capabilities. Thus the generic, fixed portion of 
a dynamic agent, together with its application programs. 



28 



29 acts as an autonomous problem solver. This means that 

30 generic capabilities (such as reasoning) are fixed at the 

31 point of startup of an agent. For agents that persist 

32 over a long period (months and years, rather than hours 
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1 or days) this means that it: is impossible to intrpduce 

2 jiew^ generic capabilities as they become available, 
3 

4 Finally, the US2002062334 does nor explain how existing 

5 functionality might be replaced- Crucially , if demands 

« • 

S are made of a carried action, and those demands pile up, 

7 and the carried action is then replaced, there is no way 

S of coordinating the rescheduling of those demands for the 

9 new carried action. By having a specific component 

10 ^responsible for handover during update, it would become 

11 possible to queue these demands until the new 

12 functionality is in a position to handle them. 
13 

14 Agents represent individuals, and typically persist over 

15 long periods. It is thus a very real problem that adding 

16 new generic functionality and upgrading existing 

17 functionality in deployed systems should be prohibited. 

18 As a consequence, it would be advantageous to implement a 

19 method by which even the core, generic f ^nationality can 1 

20 be replaced. A portion of the data of an agent may be 

21 retained, at the discretion of the update mechanism, to 

22 maintain coherence (so, for example, the beliefs of an 
2 3 agent may persist through an update of core 

24 functionality) - This would allow upgrading of deployed 

25 agents through a straightforward, scalable mechanism. 
26 

27 US2002062334 discloses that when multiple actions are 

2 8 carried by the same dynamic agent, they can exchange 

29 information through the object store of that dynamic 

30 agent. A firsr problem with this is that for a carried 

31 action to be able r.o use data from another module f it 

32 must be able to anticipate the availability and internal 

33 structure of that data. That is, it must have knowledge 



06-01-04 16:46 



From— KENNEDY COMPANY 



+0141226S838 



T-311 P. 07/33 F-0B8 



1 of the "interface" provided by that action. So for any 

2 two actions that might need to interact, they must (a) 

3 know that the other action is carried, (b) know enough 

4 about that action to know what data it can provide and in 

5 what format. In this way, any 'old' carried action will 

6 be unable to exploit the functionality developed in 'new' 

7 carried actions that was unanticipated at the time of the 

8 development of the 'old' carried action. This is a clear 
■9 theoretical problem that translates into a large 

10 practical problem in situations where agent systems are 

11 deployed on a wide basis, particularly if they are 

12 persistent, and subject to rigorous quality of service 

13 requirements (such that they can't simply be 'switched 
Id off for an upgrade) - ft. second problem, in a similar 

15 vein, is that old actions cannot know of the methods 

16 (i.e. published procedures) of a newer carried action. It 

17 is prssumably for this reason, coupled with the fact that 
IB it is much more difficult to design all possible method 

19 interfaces on a system-wide basis at the outset, than it 

20 is data interfaces, that means that the disclosed sysrem 
2L does not support method calls between carried actions. 

22 This reduces the potential for synergistic interplay 

23 between carried actions, and can lead to requiring 

24 redundancy {re implementing identical functionality in 

25 different carried actions) . The problem is rhus that one 

26 carried action cannot make a call upon the abilities of 

27 another. This violates key principles of modern 

28 programming practice aimed at code reuse and adaptation, 

29 by forcing modules to implement all the functionality 

30 that they will need, rather than utilising functionality 

31 they may know is already available within other modules 

32 in the agent. Agentative representation in mobile 

33 services represent a domain in which persistency is 
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6 

1 preferably said method means performs said function 

2 responsive to said - request - 
3 

4 Preferably said request from a requesting module 

5 comprises a label specifying a function and sa.id method 

6 means in said dockable module corresponds to the 

7 specified function. 
8 

9 Preferably said intermodule communication means comprises 

10 a store of labels and associated modules. 
11 

12 Preferably said store of labels and associated modules is 

13 a table. 
14 

15 Optionally said method means performs the funcuiOTi of 

16 docking said dockable module with said agent. 
17 

18 Optionally said method means performs the function of 

19 registering a label with said intermodule communication 

20 means, the label specifying a function supported by said 

21 dockable module- 
22 

23 Optionally said method means performs the function of 

2 4 undocking said dockable module from an agent , 

25 

26 Optionally said method means performs the function of 

27 requesting the discarding of a label from the intermodule 

28 communication means, the label specifying a function 
2 9 supported by said dockable module, 

30 

31 Optionally said method means performs the function of 

32 updating said dockable module within said agent. 
33 
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According to a second aspect of the present invention, 
there is provided a method of deployment a module in an 
agent comprising the steps of: 

• receiving a request for deployment of said module; 

• maintaining registration information relating to a 
function supported by said module; and 

• invoking said function. 

Preferably said step of maintaining registration 
information comprises maintaining a store of labels and 
associated mod-ales, each label specifying a function 
supported by a module - 



14 Preferably said store of labels and associated modules is 



IS a table. 



Preferably said step of maintaining registration 
information comprises the step of registering a label 
specifying a function supported by said module* 
This type of deployment is docking. 

Alternatively said module is a docked module and said 
step of maintaining registration information comprises 
the step of unregistering a label specifying a function 
supported by said docked module. 
This type of deployment is undocking. 

Alternatively said module is a replacement module and 
said step of maintaining registration information 
comprises the steps of s 

* unregistering a iabel specifying a function 

supported by a docked module from said agent; and 



< 
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1 » registering a label specifying a function 

2 supported by said replacement module with said 

3 agent 

4 anC i c^id method further comprises the steps of: 

5 v storing information related to the st4te of said 

6 docked module; and 

7 * instantiating said replacement module using- said 

8 stored information, 

9 This type of deployment is updating. 

10 

11 Optionally, said step of maintaining registration 

12 information further comprises the step of queuing calls 

13 to the docked modules 1 registered labels, prior to the 

14 step of registering a label specifying a function 

15 supported by said replacement module , 
16 

17 According to a third aspect of the current invention said 

18 function supported by said module according to the second 

19 aspect is the method of deployment of said module 

20 according to the second aspect. 
21 

i 

22 In order to provide a better understanding of the present 

.23 invention, an embodiment will now toe described by way of 

24 example only and with reference to the accompanying 

25 Figures, in which: 
26 

27 Figure 1 illustrates, in schematic form, an agent and a 

28 module in accordance with a preferred embodiment of the 

29 present invention,* 
30 

31 Figure 2 illustrates, in schematic form, an overview of 

32 agentative representation in a multi-service environment; 

33 
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1 Figure 3 illustrates, in schematic form, the process by 

2 which a label is resolved in accordance with a preferred 

3 embodiment of the present invention; 

4 

5 Figure 4 illustrates, in schematic fonn, module docking 

6 in accordance with the present inventions 

7 

8 Figure 5 illustrates, in schematic form, module updating 

3 in accordance with the present invention? 
10 

11 The inventions are an architecture and methods for 

12 dynamic deployment of modules in a software agent. 

13 

14 Although the embodiments of the invention described with 

15 reference to the drawings comprise computer apparatus and 
IS processes performed in computer apparatus, the invention 
17 also extends to computer programs, particularly computer 
IB programs on or in a carrier, adapted for putting the 

19 invention into practice- The program may be in the form 

20 of source code, object code, a code of intermediate 

21 source and object code such *g in partially compiled form 

22 suitable for use in the implementation of the processes 

23 according to the invention. The carrier may be any 

24 entity or device capable of carrying the program. 
25 

2 6 For example, the carrier may comprise a storage medium, 

27 such as ROM, for example a CD ROM or a semiconductor ROM, 

28 or a magnetic recording medium, for example, floppy disc 

29 or hard disc. Further, the carrier may be a 

30 transmissible carrier such as an electrical or optical 

31 signal which may be conveyed via electrical or optical 

32 cable or by radio or other means* 
33 
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10 

1 When the program is embodied in a signal which may be 

2 conveyed directly by a cable or other device or means, 

3 the carrier may be constituted by such cable or other 

4 device or means • 
5 

6 Alternatively, the carrier may be an integrated circuit 

7 in which, the program is embedded, the integrated circuit 

8 being adapted for performing, or for use in the 

9 performance of, the relevant processes- 
ID 

11 With reference to Figure 1, the architecture 100 of an 

12 agent according to the present invention is best 

13 visualised as including a torus. On the inside of the 

14 torus 102, a special module,, the core module 104, 

* 

15 attaches itself- On the outside of th^ torus, any number 
IS of application specific modules 106 , 109 fflay also become 
17 attached- The security and unity of the agent is also 

IS conceptually protected by a thin sphere 110 encompassing 

19 all the modules. The torus itself coordinates all 

20 communi cation between modules and between modules and 

21 cores this is the Inter Module Communication Layer 

22 (IMCIL) „ 
23 

24 The modular architecture is implemented as a class that 

25 defines a module, from which specific classes, 

26 corresponding to modules with specific functionality, can 

27 be derived* A module 112 is shown with its components 

2B depicted. The module contains a number of components for 

29 implementing functions : 

3Q 1. A docking method 114 that is called when the 

31 module is introduced to the agent and that describes 

32 the functionality provided by that module; 
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1 

9 



5 
6 



11 

2. An undocking method 116 that is called when the 
module is required to detach itself from the agent 

3 and terminate? 

4 . 3. An update method 118 that is called when the 

module is to he updated with a new version; and 
4 . k message handling method 120 for between-agent 

7 communication. 
8 

9 The docking method of a particular module provides the 
X0 IMCL with a list of labels. The extensible set of these 

11 labels is consistent system-wide, functioning as the 

12 definition of the system's programming interface and is 

13 maintained centrally by the service provider. A 

14 particular module implements methods that match some 

15 (typically small) subset of all labels- On docking, a 

16 module informs the IMCL which labels it provides 

17 functionality for. As each module docks, the IMCL builds 
IB a table 122 of labels against which are listed the names 

19 of the methods that provide the functionality and the 

20 names of the" modules implementing those methods . 

21 

22 In some oases, the labels implemented by a particular 

23 newly docking module may already be implemented by 
methods extant in the agent. So for example functionality 



24 

25 f implemented by method x in module ml may mean that when 

26 ml docks in an agent, it needs to register the label f 

27 with the IMCL. But some other method, y, in some other 

28 module, m2, that is already docked, may also implement 

29 functionality f . Thus the IMCL would list functionality 

30 f has -being implemented by m2.y. 
31 

32 In such situations, the IMCL adds the new label and 

33 associated method to its table, but also includes a 
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12 

1 priority ranking, applied to both the new and old methods 

2 that implement the functionality. This priority is 

3 determined on the basis of some algorithm- Recency is a 

4 good example/ whereby newer methods are always given a 

5 higher priority, but there are many such algorithms. 

7 In addition, a module's docking method calls functions 

8 provided by the core's module management. These calls 
3 register information about the module with the core, 

10 including its name, provenance and labelled 

11 functionality. The processes of docking and module 

12 updating are described below with reference to Figures 7 

13 and 8, 
14 

15 A user interacts with the electronic world for a host of 

16 reasons in a wide variety of domains; entertainment, e- 

17 commerce, professional,, and so on- The present invention 

18 provides a means of bringing together all of these tasks 

19 and domains r and providing a single, point of contact for 

20 the user, and allowing the sharing of user data between 

21 these different application domains. This contact; is the 

22 user's agent, both in the computer-science sense (where 
2 3 agent oriented programming has particular restrictions, 

2 4 techniques and approaches, and places particular demands 

25 on software), and also in the intuitive sense of 

2 6 providing services of advocacy and representation. A 

27 user's agent is their permanent representative in the 

2 8 electronic world. Ideally, each user has exactly one 

29 agent, and a user's agent represents exactly one user {at 

30 the very least, such a relationship exists in a given 

31 context) • The overall picture is as in Figure 2* 

32 
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1 ttith reference to Figure 2, an overview of agentative 

2 representation in a multiservice environment is shown- 

3 The user 202 connects to their agent 206 at any time via 

4 any device (2G phones,, multimedia mobile handsets r 

5 internet:, etc* 3 in ways that are well known. The user 

6 agents 204 which represent users in the virtual world are 

7 shown. One user has a single agent 2 06 representing hisn 

8 or hfer in all their interactions in the virtual world* 

9 The service agents 203 provide specific services to any 

10 agents that request thenw or that the service agents 

11 themselves decide to service- Information exchange 

12 between user and service agents can be initiated from 

13 either end. some service agents 210 encapsulate existing 



14 legacy services (e.g., databases, Web Services and 

15 proprietary data handling systems} . Broker agents 212 

16 can mediate between a user and service agents. The user 

17 agents service agents and broker agents, may be provided 

18 as a trusted service by a telecommunications operator. 
19 

20 An agent is a software entity with particular 

21 characteristics. We refer here to software processes that 
2.2 are: 

23 (i) persistent (in that they continue to exist for an 

24 extended real time period, adapting to a single user 

25 over that time) ; 

26 (ii) proactive (in that they include not only reactive 

27 behaviour, but also independently determined 

28 behaviour] ; 

29 (iii) communicative (in that they communicate with 

30 other agents) ; and 

31 (iv) autonomous (in that they typically cannot be 

32 directly modified by- outside agencies, but must 
3 3 instead be altered through aoinniunciation) . 



L 
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The user can communicate with his agent across 
heterogeneous networks from a variety of devices T 
including mobile handsets and internet clients. In 
addition^ however, the framework of the present invention 
supports the transparent filtering of information 
according to the device to which it is« being sent. Thus 
the components within an agent that initiate 
communication with a user ne,ed not have any 
representation of the device typ& a user is employing. 
The content of the message is instead dynamically 
tailored to the user's device (e.g. summary text to an 
SMS-enabled mobile device, still pictures to a MMS- 
enabled mobile device r streaming video to broadband 
internet client platform, etc.). 

The core is responsible for tailoring information to the 

device that is known to current ly be available to the 
user- Thus, tailoring happens independently of the 

module calls, so that individual modules do not need to 

maintain device-specific information. 

This filtering is achieved through a module-independent 
communication object that is filled in by individual 
modules when they need to communicate with the user. 
This object has. subparts for different forms of media 
(text, picture, video, audio, etc,,), A module fills in 
as many of these subparts as it is able. The core then 
mediates the sending of that message to the user, by: 
(i) identifying which device the user is currently 

employing (using a cOirtbination of historical usage 
patterns, presence information, and most recent- 
communication data) ; 



1 
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1 User data {e.g., address; credit card details? age} and 

2 laser preferences (e.g., policy on releasing credit card 

3 details; preference for aisle or window seat on planes; 

4 preferred DVD supplier) are stored in a local , private, 

5 secure database* Both user data and user preferences are 

6 extracted in three ways. First, through an eKplioit 

7 online interface that requests input on date of birth, or 

8 supports update to reflect change of address- Second, if 

9 the ag«snt recognises information that it needs from the 

10 user, it can ask for it directly [e,g, asking a yes /no 

11 question by SMS) . Third, as the user interacts with 

12 aervices manually,, the agent can intercept information 

13 either explicitly or implicitly* If the user answers a 

14 particular question from a particular online service, the 

15 agent may either store that answer for future use, or ask 
IS the user explicitly if such storage is appropriate or 

17 useful. When acting autonomously, the agent provides 

18 information that external service requires {and no more), 

19 less anything that the user has placed a restriction on. 

20 Thus, for example, when interacting with an online 

21 newspaper, the newspaper provider may request user 

22 registration, tout not demand it. In this case, the agent 

23 would provide no user information. Alternatively, when 

24 interacting with a book e-tailer, the e-tailer may 

25 require personal details including credit card data- If 

26 the user has instructed his or her agent not to give out 

27 credit card details without confirming it first, the 

2 8 agent would halt interaction with that site until user 

29 confirmation was sought and agreed. 

30 

31 These components could toe represented by the steps: 

32 1, Agent has goal of interacting with a service 
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1 2« Select required information from the user model 

2 (UM) (accesses the UM) 

3 3. Check that the user model permits all this 

4 info:raiation to be freely given (accesses the UM) 

5 If so, 

6 4, Information given to the service 

7 Otherwise 

8 5, Process tha restriction (either by terminating, 

9 or by asking the -user, or by performing some 
10 other action) 

11 



X2 The core also includes a subsystem responsible for 

13 passing messages to, and receiving messages from the 

14 user. The user may connect to his or her agent through a 

15 number of different channels: using a web browser on a 

16 PC, using a rich media mobile device (a Java phone, for 

17 example) r using a high capacity mobile device J such as 

18 one that uses GPRS) , or using an older, limited media 

19 device (say that can only handle voice and SMS traffic) - 

20 The core implements' labels that handle communication to 

21 and from such devices quite transparently: the calling 

22 module does not need to specify the different 

23 communication types at all - 
24 



25 The means by which one agent communicates with another is 

2 6 implemented in the core. Rather than supporting only 

27 agent-to-agent messages r the architecture is instead 

2 8 built around the idea that it is individual modules 

29 within agents that communicate with one another {this is 

30 "between agent module-module" or BAMto communication) * 

31 Thus a module with expertise in buying in a particular e- 

32 commerce institution will communicate with a module in 

33 another agent that has expertise in selling in that same 
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1 labels with the IMCL and initialise 728 the BAMM handler 

2 before ending the process 730. 
3 

4 The undacking method of a particular module, informs the 

5 IMCL and the core module management that the 

6 functionality provided by the module is terminating r and 

7 then kills the module's thread. 

8 ■ 

9 One of tha advantages of decoupling of functionality in 

10 separate modules is in allowing modules to be added to an 

11 agent on~the fly- Such modules need not have 

12 functionality that ia known before the agent is 

13 instantiated, as each module describes its own behaviour 

14 and functionality on docking- The same approach can be 

15 used to update existing modules with new versions* 

16 

17 Another, practical, advantage of the approach is that it 

18 removes compile time dependencies: a module developer can 

19 design, implement and rest a module which makes calls to 

20 another module that they do not have, or do not have 

21 access to, or r indeed, chat has not been developed at 

22 all. This simplifies many of the problems of software 

23 engineering in the large f and of multi-site collaborative 

24 development work. 
25 

26 The core is responsible for the handling of new modules 

27 and module updates. It arranges for outgoing requests to 

28 be sent for new modules (or for updated versions of 

29 existing modules) f handles incoming requests for the 

30 agent to take on new modules* and handles the process by 

31 which a module is added. So, for example, if the user 

32 decides to sign up for a new, free service,, the core may 

33 send a message to the coordinating agent for that 
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1 service, requesting the module. After some brief 

2 negotiation, the coordinating agent may send back a 

3 request to install the new module. The core module 

4 management accepts and downloads the software. After 

5 updating the internal module database r it starts the new 

6 module, which then docks with the IMCL, as described 

7 above - 
8 

9 The process of updating a module and the handover between 

10 the older and newer versions in illustrated in Figure 5. 

11 With reference to Figure 5, a flowchart 8 00 of the module 

12 updating process is shown. First, either a uses: specifies 

13 802 a module, the core receives a message 804 specifying 

14 a module to update, or the pore determines 80 6 that a 

15 module should be updated* The core reasons and decides 

16 808 whether or not to update the module. If an update is 

17 determined to be completed, the core calls 810 the IMCL 

18 with the gstmodule label and URI specifying the location 

19 of the Java Archive file. If not, the updating process 

20 ends 836, The IMCL then resolves 812 upda temodule label 

21 to the ModuleManagement . ModuletJpdate method in the core 

22 and the IMCL calls 814 the core's 

23 ModuleManagement .ModuleGet method with the module name 

24 and the URI- ModuleManagement calls 816 the module's 

25 UpdateMe method and the UpdateMe method returns the 

26 module state information, which is then stored 818 by 

27 ModuleManagement. ModuleManagemewt downloads 820 the Java 
2 5 Archive file from the URI. The .jar manifest file 822 

29 specifies the Module class and the Module class 824 

3D includes the DackMe method* With this information, the 

31 ModuleManagement instantiates 826 the Module class and 

32 calls 828 the DockMe method with the stored state 

33 information- The Dockwe method may then register 830 
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1 labels with the IMCL and initialise 832 the BAMM handler 

2 before the Module Management calls 834 the UnDockMe 

3 method on the old module and unloads the class then ends 

4 the process 836. 
5 

6 The core's, module management also handles more complex 

7 situations, in tthich a new version of an existing module 

8 is installed- The handover between the modules requires 

9 careful timing to ensure service to the rest of the agent 

10 are not disrupted. When a new version of a module is to 

11 be docked, the core module management calls the old 

12 version's update method, and, at an appropriate moment 

13 {at a break between discrete tasks that the module is 

14 working on) the module returns a represent ion of its 

15 current state. 
16 

17 Then, as part of the new module's docking method, ±t 

18 accepts the state of the previous version's data, so that 

19 it can pick up from where the previous module left off. 
20 

21 Dur±ng the handover between versions of a module, the 

22 IMCL queues calls to the modules 1 labels , awaiting the 

23 docking of the new version. 
24 

25 Some of the functionality described herein is collected 

25 together in the core- Rather than hardwire a distinction 

27 between core and other modules, the core is instead 

28 implemented in just the same way as any other module. 

29 This me<ans thar calls on core functionality can exploit 

30 the robustness of the label-based, IMCL-inediated method 

31 calling. 
32 
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1 Xt also means that core Itself can be updated and docked 

2 through just the same process as any other module {so the 

3 activity of the module* management component in the old 

4 core is interrupted and then taken up by the activity of 

5 the module management component in the new core) - This 

6 means that new core functionality can be deployed 

7 dynamically, without other modules needing to know ahead 
B of time all that might be included - 

10 Further modifications and improvements may be added 

11 without departing from the scope of the invention herein 

12 described* 
13 
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